Inventions described herein relate to a novel, corridor-based framework that performs threat assessment and provides varying degrees of mutually consistent automated operator assistance as a threat response in human-machine systems, such as locally or remotely-operated passenger vehicles, transport vehicles, agricultural machinery, fork lift trucks, aerial vehicles, robots, or surgical tools. This framework explicitly considers human and machine dynamics without presuming operator intentions or limiting the avoidance maneuver (and its associated threat assessment) to a specific path. It provides a unified framework that allows for various modes and levels of mutually consistent operator assistance, from operator warning to stability control to passive intervention, to active semi-autonomous control, and finally, to autonomous machine operation.
Automotive active safety systems are concerned with preventing accidents through the introduction of various computer-controlled actuation methods to inform, improve, or override a human operator's steering and/or braking performance. Active safety systems currently in existence include yaw stability control, roll stability control, traction control, and antilock braking, among others. While these systems reduce accident frequency, their path-based and largely reactive nature limits their ability to: 1) accurately assess the threat posed by a given scenario and 2) adequately determine when and how to intervene to assist the driver. This dependence on a specific trajectory (amidst a myriad of options available to the operator) reduces the accuracy and significance of the threat assessment and leads to controllers that selectively replace (rather than assist) the driver in order to follow an automation-designated path.
The inventions described and claimed herein relate, primarily, to the unified nature of the framework, and its flexibility in being able to accurately assess threat, and then participate in one or more of a wide variety of mutually consistent operator assistance modes, of varying levels of operator autonomy, from complete autonomy, to no autonomy. Inventions described and claimed in the Threat assessment application relate primarily to threat assessment aspects of this novel framework. Both have similar disclosures.
A basic premise of threat assessment for such assisted and automated systems is generally as follows. First, sensing systems such as radar, LIDAR, cameras, inertial measurement units and GPS localization systems are used to detect, classify, and track the position of objects and the drivable road surface in the host vehicle's vicinity as well as measure vehicle states. Once these potential hazards have been identified, localized and their motion has been estimated, a threat metric is used to quantify the threat they pose to the host vehicle, together with the threat of departing the drivable road surface due to loss of vehicle control. As used herein, threat assessment or threat prediction is used to mean identifying hazards and quantifying threat. Many threat assessment technologies are designed to then trigger and/or implement countermeasures to reduce the threat. These countermeasures can be passive or active. The effectiveness of threat assessment metrics depends on their ability to correctly identify hazards and accurately assess the threat that potential hazards pose to the host vehicle.
Threat metrics described in the literature predominantly use time-based, distance-based, and deceleration-based measures to characterize the threat level of a given scenario. Time-based threat measures project time to collision (TTC) based on current speeds, positions, trajectories, and (in some formulations) other vehicle states. Distance-based metrics are generally calculated using prevailing range and vehicle speeds and require constant velocity/acceleration assumptions and simple hazard geometry. Finally, acceleration-based metrics assess the threat of a given maneuver based on the minimum (and often assumed constant) lateral or longitudinal acceleration that a simple avoidance maneuver would require, given the current position, velocity, and acceleration of both host and hazard. In another approach, estimate is made of the lateral acceleration required to execute a constant radius evasive maneuver. That implementation then compares this acceleration to a threshold value. When the required acceleration reaches this threshold, braking countermeasures are implemented to reduce the vehicle's longitudinal velocity.
While the above threat metrics have been shown to provide useful estimates of the danger posed by a given maneuver, they suffer from many drawbacks. They are not well suited to consider multiple hazards, complex vehicle dynamics, or complicated environmental geometry with its attendant constraints. The geometrically-simple (straight-line or constant-radius-turn (CRT)) avoidance maneuvers assumed by these metrics may also misestimate the true threat posed by scenarios where the optimal avoidance trajectory follows a curve of varying radius or non-constant velocity/acceleration.
At least one known method of assessing threat relates to a vehicle that is intended to navigate along a path. The path may be predetermined, or calculated, based on data, such as information about obstacles and a path followed by a track, such as a roadway in the case of a road vehicle, such as an automobile. The path is a curve of simple geometry, having essentially no width. If the vehicle deviates from the zero width path, the system determines that danger has arisen, and the system generates a threat signal. However, in fact, the threat of actual danger is potentially low, because vehicles driven by a human operator typically operate within a field of safe travel, or a corridor, rather than along a relatively arbitrary line, such as the centerline of a roadway. This dependence on a specific trajectory (amidst a myriad of options available to the operator) reduces the accuracy and significance of the threat assessment and leads to controllers that selectively replace (rather than assist) the driver in order to follow an automation-designated path.
The following terms will be used herein as follows. A path is a simple geometric curve in two-dimensional space, along which a vehicle may travel. The path, in x-y space may be defined by a function, y=f(x). A path has a width of essentially zero. A trajectory is a physically-achievable and time-parameterized sequence of vehicle states (such as velocity, yaw angle, wheel sideslip angle, etc.) over a time horizon, By physically-achievable, it is meant that for every trajectory, there exists a set of controller inputs, such as braking torque and steering angle, that when applied to a model of the vehicle produce the desired trajectory. It has been mentioned that the trajectory includes the velocity of a vehicle as an element. It may also be thought of as having time as a parameter of the path, which may then establish velocity at different locations.
A corridor is a swath through two-dimensional space, which may be defined by an inequality ymin<y<ymax, where each of ymin and ymax are themselves defined by ymax=g(x) and ymin=h(x). Thus, a corridor may be considered to be the space between two curves in two-dimensional space. Travel anywhere within the corridor is considered to be safe. A region is a concept that is defined in connection with inventions hereof, and it will be defined below.
A Model Predictive Controller (MPC) is an optimal control method typically used to generate an optimal set of control inputs (spanning through a future time horizon) required to track a desired path while minimizing a user-defined objective function. In typical usage, only the first element of this command input sequence is implemented at each solution timestep and the remainder are disregarded.
A human driver typically operates a vehicle within a safe range of vehicle states. For example, the driver typically maintains the vehicle's position on the roadway (lateral position state) within the corridor defined by road or lane edges. Similarly, other states such as vehicle velocity, lateral acceleration, etc. provide some indication of threat to the driver, who (consciously or subconsciously) seeks to keep them within a safe operating range (or between upper and lower bounds). Depending on the driving conditions, posted speed limits, and other factors, for example, a driver may allow vehicle velocity to vary between 53 and 70 miles per hour rather than slavishly maintaining 55 miles per hour along a roadway. Likewise with vehicle sideslip, which the driver will typically strive to maintain within a reasonable (or safe) range. Thus, a human operator operates within an N-dimensional region of state space, rather than along a simple, zero, or nearly-zero width curve of a physical trajectory.
In many cases, it may be desirable to assess threat based on a realistic roadway corridor than an unrealistic single path on the roadway. Similarly, it may also be desirable to assess threat based on a realistic N-dimensional region of state space (as explained more fully below, which includes as a portion, the two-dimensional corridor).
Some known threat assessors exist as a separate system, not integrated with other systems of the device analysis and control apparatus. These systems may base the assessment of threat on a device state exceeding a relatively arbitrary threshold. Further, these threat assessors typically do not provide the control input necessary to decrease the threat they've assessed.
Other threat assessment approaches use only rudimentary threat assessment metrics based on, for example, the current deviation of the device from a predetermined optimal path. It would be desirable to be able to take advantage of predicted future states of the device in assessing threat.
Still other threat assessment approaches also do not consider combined effects of vehicle dynamics, stability constraints and terrain interactions to evaluate maneuver severity. However, it would be desirable to be able to consider these matters in threat assessment.
Thus, an object of inventions disclosed in the Threat Assessment application is to take advantage of predicted future states and predicted future optimal inputs to assess threat. A further object would be to be able to consider combined effects of vehicle dynamics, stability constraints and terrain interactions to evaluate maneuver severity in assessing threat. Still another object would be to be able to assess threat on a realistic corridor, rather than an unrealistic single path. Another object would be to assess threat with an apparatus that is integrated with other systems of system analysis and control, which are also used to control the device or system, rather than assessing threat on a device state exceeding an arbitrary threshold. Still another object would be to use the assessment of threat to generate an operator assistance signal to assist the operator in safely operating the device or replace the operator as necessary to ensure safe operation of the device.
Known device control systems use distinct and sometimes competing modules to assist an operator. Thus, they often provide conflicting and inconsistent operator assistance threat responses.
These and other objects of inventions disclosed herein will be more fully explained and understood with reference to the Figures of the Drawing, which are: